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C^ Abstract 



^— «, In this paper, we consider the weighted graph matching problem. Recently, approaches to this problem based on 

fvj spectral methods have gained significant attention. We propose two graph spectral descriptors based on the graph 

Laplacian, namely a Laplacian family signature (LFS) on nodes, and a pairwise heat kernel distance on edges. We 
show the stability of both our descriptors under small perturbation of edges and nodes. In addition, we show that our 
pairwise heat kernel distance is a noise-tolerant approximation of the classical adjacency matrix-based second order 
. compatibility function. These nice properties suggest a descriptor-based matching scheme, for which we set up an 

^ integer quadratic problem (IQP) and apply an approximate solver to find a near optimal solution. We have tested 

^ our matching method on a set of randomly generated graphs, the widely-used CMU house sequence and a set of real 

'^^ images. These experiments show the superior performance of our selected node signatures and edge descriptors for 

r \ graph matching, as compared with other existing signature-based matchings and adjacency matrix-based matchings. 

,'^, 1 Introduction 

■^ Graph matching techniques have been widely used in computer vision such as 2D [29, 33] and 3D [ ' '] image analysis, 

K^ biomedical identification [20, '*'], and object tracking over video sequences [4, 28]. Most of the algorithms used in 

practice belong to the category of inexact matching where people consider graphs with noise. In this paper we consider 
the problem of graph matching for weighted undirected graphs with noise. These kind of graphs appear commonly in 
computer vision when objects in images undergo either rigid or deformable motion. 

In this paper we propose to use spectral descriptors as first and second order compatibility functions in an integer 
^\ quadratic programming (IQP) setting. Spectral methods [ ), 6, 9, 12, 34, 8] have been widely used in graph matching 

^f) algorithms because of the fact that the graph adjacency matrix and its derivatives are spectrally invariant under permu- 

^-H tations. Recently, spectral node signatures have drawn significant attention for shapes, e.g. the heat kernel signature 

►;>. (HKS) [""] and the wave kernel signature (WKS) [ ']. Inspired by the common structure of HKS and WKS, we pro- 

pose in this work a generic one-parameter family of node signatures based on the graph Laplacian and generalizing 
the HKS and WKS, which we call the Laplacian family signatures (LFS). This generic structure permits other spectral 
J3 node signatures to be easily generated. In addition, we unify the stability of HKS and WKS under small perturbations 

of the graph into our LFS framework. 

Besides the use of LFS as the first order compatibility condition, we propose to define a pairwise heat kernel 
distance as a second order condition. We show in the paper that the pairwise heat kernel distance is a noise-tolerant 
approximation to the classical pairwise adjacency distance, which is often used as the second order compatibility 
function [ ]. 

Setting LFS as the first order compatibility condition and pairwise heat kernel distance as the second order compat- 
ibility condition, we formulate the graph matching problem as a general IQP. We use the approximate solver proposed 
recently by Cho et. al. [ ] — a reweighted random walk matching algorithm that has shown superior performance 
over other state-of-the-art IQP solvers. We test our spectral descriptors on three different graph matching experiments, 
i.e. randomly generated graphs, CMU house sequence, and real images. Our proposed descriptors are more robust 
than the classical adjacency matrix based matching. 



The rest of the paper is organized as follows. In Section 1.1, we review previous work on graph matching that 
is related to our proposed descriptors. We introduce our spectral descriptors and their properties in Section 2. The 
descriptors are then used as compatibility functions and set up as IQP in Section 3. Last but not the least, we present 
our experimental results in Section 4 before the paper is concluded in Section 5 

1.1 Related Work 

Node-based signatures have been used on graphs, both inside and out of matching. Joilli et. al.[ ] proposed a 
signature composed of the degree of the a node followed by the ordered weights of each incident edge and padded 
with zero if necessary. Gori et. al. [12] constructed node signatures from the steady state distributions of simulated 
random walks similar to PageRank. Eshera [ ] built signatures for attributed relational graphs (ARG). Shokoufandeh 
et. al [ ] constructed feature-based node signatures for bipartite matching. The same authors [ ] later proposed a 
topological signature vector (TSV) for directed acyclic graphs (DAG). Outside of graph matching, Wong et. al [35] 
proposed a topological signature. Their signature is constructed as a local histogram of the number of nodes. 

Our signature is also related to spectral method in graph matching. Among the pioneering work of spectral method 
is Umeyama's [ ] weight graph matching algorithm. His method was later generalized to graphs of different size 
[18, 3S]. Robles-Kelly et. al. [ ] ordered the nodes from the steady state of markov chain with the edge connectivity 
constraint and matched using edit distance. In their 2005 paper [ ], node ordering was done using the leading 
eigenvector of the adjacency matrix. Qiu et. al [ ] considered using Fiedler vector to partition the graph for a 
hierarchical matching. Their method, however, works only on planar graphs. Cho et. al. [ ] constructed a reweighed 
random walks similar to personalized PageRank on the association graph with the addition of an absorbing node. They 
computed the quasi-stationary distribution and discretize the continuous solution to find a matching. Emms et. al. [9] 
built an auxiliary graph from the two graphs and simulate a quantum walk. Particle probability of each auxiliary node 
is used as the cost of assignment for a bipartite matching. 

In a broader sense of relatedness to our work is other relaxation-based matching algortihms. Gold and Rangarajan 
[II] proposed the well-know Graduated Assignment Algorithm, van Wyk et. al. [ ] designed an projection onto 
convex set (POCS) based algorithm to solve IQP by successively project the relaxation solution onto the convex 
constraint set. Schellewald et. al. [ : ] constructed a semidefinite programming relaxation of the IQP. Leordeanu et. 
al. [ ] proposed a spectral method to solve a relaxed IQP where they drop off the linear inequality constraint during 
relaxation and only incorporate it at the discretization step. The idea was further extended by Cour et. al. [ ], where 
they added an affine constraint during relaxation. Zaslavskiy et. al. [3 ] approached the IQP from the point of a 
relaxation of the original least-square problem to a convex and concave optimization problem on the set of doubly 
stochastic matrices. Leordeanu et. al. [ ] proposed an integer projected fixed point (IPFP) algorithm to solve the 
quadratic assignment problem. 

2 Graph Spectral Descriptors 

In this section, we will introduce the Laplacian family signatures (LPS) as graph node descriptor and the heat kernel 
as the graph edge descriptor and their properties. 

2.1 Preliminary on Graph Laplacian 

Let the undirected graph under discussion be denoted by G = {V,E), where V is the set of vertices, and E C V x V 
is the set of edges. Let w be the weights on edges, i.e. w : E ^^ M+. The graph Laplacian is defined ?& C = D — A, 
where A is the graph adjacency matrix with 

^'' 1 otherwise 

and £> is a diagonal matrix of total incident weights, i.e. Da = ^ Aij. 



£ is symmetric and positive semi-definite and has non-negative eigenvalues, the smallest of which is and the 
multiplicity equals the number of connected components. 

Similar to classical Fourier transform, where complex exponentials e'"^ are the eigenfunctions of Laplacian op- 
erator -£^, eigenvectors of C can be used to define the graph Fourier transform [13]. In that sense, if (A^, <j)i) are the 
eigenpairs of £, A^ are discrete frequencies and 0^ are eigenfunctions of each frequency. 

2.2 Laplacian Family Signatures 

Definition 1. For a graph G with laplacian C, if{Xi, (pi) are the eigenpairs of C, then the LFS si{t) of node I ^V is 
defined as 

k 

where h(t] A^^) is called the construction kernel. 

Choosing h{t; A^) — exp{~tXk) results in the heat kernel signature (HKS) [27], where the authors have shown the 
multi-scale and informative properties on manifolds with application on shape analysis. HKS came from the simulated 
heat diffusion process. For each node, the signature captures the amount of heat left at the node at various time slices 

when a unit amout is put on the node initially. While selecting h{t: Afe) — exp(— ^ ^2^^ '' )' ^^ becomes the wave 
kernel signature (WKS) [ ]. WKS came from the natural counterpart of heat diffusion, the particle quantum process. 
The signature captures the particle distribution probability at different energy band. If assuming h{t; Xk) = 9{tXk) 
is band-pass with special behavior at origin, the kernel becomes wavelet kernel [ 1 3] which could be easily extended 
within our generic framework to wavelet signature. 

The family of signatures are intrinsic, namely they are invariant under isomorphism. Besides, one important 
property of LFS is its stability under small perturbations. HKS and WKS have been shown to be stable in their original 
work. Inspired from them, we unify the stability to LFS and extend it to Laplacians with repeated eigenvalues, (see 
supplementary material for the proof). 

Theorem 1 (LFS Continuity). Let C be the laplacian of a graph, and C be a perturbed laplacian. Let the size of the 
graph be n, and Xi < ■ ■ ■ < X^ denote the k distinct eigenvalues of C Let Si{t) and s[{t) denote the LFS's of node i. 
//Aj+i - Aj > S,yj, \\C - C'Wf <e<6, and h{-; ■) e C^iRl), then ifk = n, we have 

\m-s,{t)\<Co{5,t)t, 

where Co{S, t) is a constant independent of e, and if k < n we have 

where Ci {t) , C2 [t) are constants depends only on t. 

When the number of nodes are different, we could use lifting ideas similar to [""-] to prove its stability. 
To better illustrate the stability of LFS, Fig. 1 a showed the WKS of all the nodes from a randomly generated graph 
with 5 nodes and perturbed counterparts. 

There are two signature distances defined in [ ] and [ ] respectively. 

Definition 2. Let imin = t^ < ti < ■ ■ ■ < ts — imax be discrete samples. Let Si (t) and Sj (t) be two LFS. Then the 
relative Li distance [3] between two LFS is defined as 



di{i,j) 
and the L2 distance [- /] is defined as 



sr^ \si{tk) - Sj{tk)\ 

^ S^itk) + Sjitk) ' 



d2ihj)= El*»(*fc)--'*j(^fe)l' 
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(a) WKS for a randomly generated graph of 5 nodes and when 
perturbed. 



(b) HKS and WKS kernels for a randomly generated graph. 
Figure 1: Perturbation of WKS and Illustration of HKS/WKS kernels. 



L2 distance reflects more the absolute difference upon each time slice t, while relative Li distance that concentrates 
more on the relative difference on each t. 

Within the family of signature, nevertheless, HKS and WKS are representative of two categories, namely, coupled 
and decoupled LFS. The significance of coupling to matching performance is shown in Section 4. 

Definition 3. A coupled LFS admits a construction kernel of the form 

h{t;Xk) ^ n{tXk) 
for some function /i(-). 

An LFS is called decoupled if it is not coupled. Wavelet signature, therefore, is another realization of coupled LFS. 

The coupling of parameters t and A^ is one of the major differences in HKS and WKS as pointed out in [3] that 
would affect their matching accuracies (as shown in Section 4). Fig. lb showed the construction kernel of HKS and 
WKS for a randomly generated graph with 5 nodes. The decoupled WKS kernel makes the sampling more effective 
as for each discrete frequency (eigenvalue), there is a kernel which is dominating all other kernel within a small 
neighborhood around that frequency and as a result each frequency is well represented in the signature. While for the 
coupled HKS kernel, one kernel (exp(— tAi)) uniformly dominates all other kernels over the domain, which makes 
high frequency components less and less representative in the signature. 

On the other hand, the Gaussian function used in WKS kernel per se is locally flat at the origin (first order derivative 
is zero), and therefore more tolerant to eigenvalue perturbations. 

2.3 Heat Kernel 

In this section, we will introduce the concept of heat kernel. The heat diffusion process on graph G is governed by 



the heat equation Lu{t) + 4j.u{t) = 0, where u(t) G W^' is the heat distribution vector, and C is the graph laplacian. 
Given an initial heat distribution / : y 1— > M'^', let Titif) be the heat distribution at time t. Ht is called the heat 
operator and can be expressed as Ht — e^*^. Both C and H are linear operators mapping real-valued functions 
defined on vertices of graph to another such function, and they share the same eigenvalues. 

Similar to heat diffusion on manifold, there is also a heat kernel on graph, which satisfies 7^4/(2:) = J^vev ^t{x,y)f{y). 
kt{x, y) can be think of as the amount of heat transferred from node x to node y at time t when unit amount is put on 
X at i = 0. The heat kernel has the following eigen-decomposition 



kt{x,y) 



\v\ 

E 



exp{-tXi)4i^{x)4>i{y), 



Stability of kt{x, y) can be shown similar to that of LFS in section 2.2. 

kt{x,y) is a low-pass filter on graph analogously to that in signal processing. If we write heat kernel ICt = 
^j exp{—tXi)(j)i(t)J, for any initial distribution / e M'^', noticing that f{i) ~ cpj f is the Fourier coefficient, Titf — 
^tf — X^i exp(— tAi)/(i)0i. For a fixed t, the scaling function exp(— tAi) suppresses high frequencies and hence JCt 
has a low-pass effect, t in that sense controls the pass-band, namely as t increases, passband decreases. 

Definition 4. Let kt and k[ be two heat kernels ofG = {V, E) and G' = {V , E'). For i,j £ V and a,b € V, the 
pairwise heat kernel distance is defined as 



and the pairwise adjacency distance is 



df{i,j,a,b) = \kt{i,j)~k't{a,b)\, 



d''it,j,a,b)^\A{i,j)~A'{a,b)\ 



The pairwise heat kernel distance df- is noise-tolerant approximation of pairwise adjacency distance d^ (see sup- 
plementary material for a proof). 

Tlieorem 2 (Noise Tolerant Approximation). Let d'f'{i,j,a, b) and d^{i,j,a, b) be the pairwise heat kernel distance 
and pairwise adjacency distance for graph G and G'. Let Xi and A^ be the set of eigenvalues ofG and G' . 



lim 

t->-o 



df{i,j,a,b) . . 
=d (i,j,a,6), 



and 



J2 J2('^f{h i, a, b)f < J2 exp(-2tA.) + ^ exp(-2a^) 



For small t, df is a good approximate of d"^. As t increases, df is smoothed out. Therefore it starts from a good 
approximation to d"^, and the differences in d-^ was smoothed away in df as t increases. That way, it becomes noise 
tolerant, because ideally without noise d^ would be zero everywhere for matched pairs. 



^^^^■^^^Pb i^^H^^^n 




(a)d^ 



(b) df @t^ 0.1 



(c) df @t = 0.8 



(d) df @t = 2.0 



Figure 2: dj^ at different time and comparison with d"^ 



Fig. 2 illustrated an example of this smoothing effect on a noisy pair of graphs. When t is small, df is a well 
approximate of d'^. While when t goes large the difference in df is smoothed away. 



3 Matching Scheme 

To compare the performance within LFS and with other node signatures, the problem is cast as a bipartite graph 
matching problem as in existing node signature based matching work [25, 12, 14], where costs are set as the distance 
between signatures (d2 for HKS and di for WKS). The problem is solved by the well-known Hungarian algorithm 
[ ] in the complexity of 0{ii?), where n is the size of the larger graph. 

In addition to node signatures, we could include pairwise heat kernel distance df as the second order constraint 
and formulate the problem in an integer quadratic program (IQP) setting. 



For two graphs G = (V, E) and G" = [V , E'). Let d{i, a) be the distance between their node signatures. We 
construct a compatibiHty matrix W G mI^II^'I><I^II^'I as 

Wia,jb - i ,/ . X . , 

If X e {0, 1}I^II^'I be the one-to-one mapping matrix and x e {0, Ijl^H^'l is the vector form of X. The IQP could 
be written as 

X* = argmin(x Wx) 

s.t. xe {0,1}I^II^'I, Vi V x,„ < 1, Va Vx„ < 1 



a(£V' i<£V 



As is well-known that this problem is NP -complete and there are a whole literature of approximation algorithms. 
Comparison of the performance of different IQP approximation solvers is outside the scope of our paper In our 
experiment, we selected a recently proposed algorithm, reweighed random walk matching (RRWM) [ ]. The main 
reason we chose this algorithm is the superior performance they have shown compared with other state-of-the-art 
approximation algorithm, including SM [.-■], SMAC [ ], HGM [ ], IPFP [. ,], GAGM [. ,], SPGM[ ]. For the 
completeness of our presentation, we Ust a peudocode of their algorithm in Algorithm 1 and left details to their 
original paper. 

Algorithm 1 RRWM [ ] 



Given compatibility matrix W, reweight factor a, inflation factor f3, 
Set Wia.jb = for conflicting matchings. 

Compute d,nax = maXia J2jb ^iajb, 

Initialize transition matrix P = W/d,nax, and x uniform, 
repeat 

x^ = x^P, 

y^ — exp(/3x/ maxx), 

repeat 

Normalize rows, y,a = y^a/ Ea Via^ 

Normalize columns, y^a = Ym/ Y.i Yia, 
until y converges 

y = y/Eyia, 

x^ = ax^ + (1 - a)y^, 

x = x/x;xiQ, 

until X converges 
Discretize x. 



4 Experiments 

We tested our descriptor on three different datasets: 1. synthetically generated random graphs; 2. CMU House 
sequence for point matching; 3. Feature matching using real images. 

4.1 Sy the tic Random Graphs 

In this section, follow the experimental protocol of [ ], we synthetically generate random graphs and perform a com- 
parative study. In the first part of the experiment, we use random graphs from Erdos-Renyi model G{n, m). For each 
selected edge, we add a uniform [0, 1] random weight. The graph is then perturbed by adding random Gaussian noise 
A/'(0, a'^) on selected edges. 



In this test, we compare the performance within our LFS and with existing nodes signatures, namely degree vector 
signature (DVS) [ ], local histogram signature (LHS) [ ], random walk signature (RWS) [ ]. Although HKS and 
WKS are the only existing LFS known in the literature, it is not difficult to construct LFS using other form of kernels. 
Noticing that both HKS and WKS kernels are continuous probability density functions (pdf) for some distribution, 
we generalize them to other form of continuous pdf's: I. Gamma kernel (iAi)*^^^ exp {—tXi/6), 2. Gaussian kernel 



exp [—{t 

5. t kernel (l + (iAOV0)"''''+'^/' 



exp (-(iA,)726'2 



^(Ai))^/2(T^), 3. Laplacian kernel exp (— 1 1 — ^(Ai)|/CT),4. Rayleigh kernel (tXi)'^ 

(iA,)Ve')"'''+^^^^ 6. Inverse Chi square kernel (iAi)~'=/^"^ exp {-e/2t\,). 
This set of kernel functions are by no means exhaustive. We hope, however, it will illustrate the difference in 
between coupled and decoupled LFS. Note that HKS is a special case of Gamma kernel and WKS is a special case of 
Gaussian kernel. 

Average accuracy of an image to the rest of the sequence 









' ^^^ 




DVS 


\ \ 




LHS 


\ \\ 




RWS 


\\ 


-- 


HKS 


^ \ \ 




- WKS 
Gamma 


i'. \ Y 


-- 


Rayleigh 






t 

invchi2 

laplacian 




S: 


Ssas. 


- ■ - - - - - ■,- - - ■ , 1 




Image Index 



(a) G(n, m) random graphs. (b) CMU House sequence 

Figure 3: Matching Performance 

We use bipartite matching as the matching scheme. In the experiment, we set n ~ 50 and to is uniformly in 
[400, 1000], and generate 100 pairs of graphs. Fig. 3a shows the average accuracy ' over the amount of noise added to 
the graph. As can be seen, decoupled LFS, WKS and Laplacian LFS, perform the best over all tested signatures. DVS 
performs the second tier, while coupled LFS, including HKS, Gamma LFS, Rayleigh LFS, t LFS and invchi2 LFS, 
are on the third tier, perform comparably with RWS. LHS is the worse in all tested signatures. This result validates 
our analysis on coupled and decoupled LFS, and the specific form of kernel h{t; Xk) seems less important than the 
coupling of parameters. 
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Figure 4: Matching accuracy of in the IQP setting. Note we used HDD in the legend of the figures to represent df. 



'Accuracy is defined as the fraction of correct matches over ground truth matches. 



In the second part of the experiment, we test different node signatures together with df as the pairwise constraint 
in the IQP setting. The random graphs are generated according to [ ] (codes are pubHcly available). For a pair of graph 



,(1) 



(2) 



Gi and G2, they share n-i^ common nodes and n^^^l and n^^^l outlier nodes. Edge weights are randomly distributed in 
[0, 1], and random Gaussian noise JV{0, cr^) are added. 

In this experiment, we test the matching performance for W of i) only d"^{i,j,a,b), ii) only d'^{i,j,a,b), iii) 
df{i,j, a, b) with different node signatures, on three different settings: 1. different level of deformation noise a; 2. 
different number of outliers; 3. different edge densities p. Fig. 4 showed the average matching accuracy. Red solid 
curve for RRWM is to match using pairwise adjacency distances d"^ only as the baseline. With d^ substituting d-^, 
the matching performance is more noise tolerant. Comparing Fig. 3a and Fig. 4 (a), it can be seen that the large 
performance gap among node signatures, however, was marginalized out because of the second order compatibility 
constraint df'. In Fig. 4 (c), the matching accuracy is much improved at different edge densities for a relatively large 
deformation noise (a = 0.5). 



4.2 CMU House Sequence 

In this experiment, we use the CMU House sequence to test our descriptors. This sequence has been widely used to 
test different graph matching algorithms. It consists of 1 10 frames, and there are 30 feature points labeled consistently 
across all frames. We build fully connected graphs purely based on the geometry of the feature points, taking the 
Euclidean distance of the key points as the weights between pair of feature points. Affinity matrices W are set up 
similarly as in Section 4. 1 . We compute the average matching accuracy of each frame to the rest of frames in the 
sequence. Fig. 3b showed the accuracy of the matching. As can be seen the matching performance was improved 
when d^ is used to substitute d-^. With WKS as the first order compatibility, furthermore, the matching accuracy 
is much improved. Fig. 5 showed an example of the matching between the first and the last frame of the sequence 
(yellow lines are correct matches and red lines are wrong matches). 
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Figure 5: Matching on House sequence. 



(c) WKS+df RRWM 



4.3 Real Image Feature Matching 

In this experiment, we test our descriptor on the real image dataset used in [5]. This image dataset consists of 30 
pairs of images with labeled feature points. In their original paper, affinity matrix W^at is built considering the 
similarity between the appearance-based feature descriptors and the geometric transformations. Let dfe^t{i,j, a, b) be 
their compatibility functions. We put our spectral descriptors as additional structural compatibility, and set up the 
affinity matrix W as 

„r }dkAi,j,a,b)+adf-{i,j,a,b) iy^j^a^b 



l3d{i, a) 



J, 



Therefore, we could evaluate how our graph-based structural descriptors would improve the matching results. In our 
experiment, we tested different combination of a and /?. If only considering df, /3 = 0, a = 1 gives the best average 
accuracy, and by adding WKS as node signature constraint, a = 0.5, /3 = 4 gives the best average accuracy. Table 1 
listed the average accuracy as a comparison. Fig. 6 showed one example of matching results. 



RRWM rffRRWM WKS+rf^ RRWM 
69.62 70.68 70.70 

Table 1 : Average accuracy of real image matching. 




(a) RRWM (36/67) (b) WKS+rff RRWM (53/67) 

Figure 6; Matching on real images. 

5 Conclusion 

In this paper, we considered weighted graph matching problem. We propose to solve this problem in a IQP setting, 
where we use LFS and the pairwise heat kernel distance as the first and the second order constraint. We showed 
their properties that would benefit matching. In the experiment, we compared node signatures within LFS and with 
other existing ones, and found that the decoupling of parameters is crucial to the matching performance. Also, the 
combination of both compatibility functions increased the matching performance over the classical adjacency based 
compatibility. 

Future work include pursuing the best signature within LFS from a learning perspective when some training dataset 
is given. 



References 

[1] Lecture notes 254a notes3a. http://terrytao.wordpress.com/2010/01/12/ 

254a-notes-3a-eigenvalues-and-sums-of-hermitian-matrices. 13 
[2] Wikipedia. http://en.wikipedia.org/wiki/Eigenvalue_perturbation. 11 

[3] M. Aubry, U. Schlickewei, and D. Cremers. The wave kernel signature: A quantum mechanical approach to 
shape analysis. In IEEE International Conference on Computer Vision (ICCV) - Workshop on Dynamic Shape 
Capture and Analysis (4DM0D), 20 11. 1, 3, 4 

[4] H. T. Chen, H. Lin, and T. L. Liu. Multi-object tracking using dynamical graph matching. IEEE Computer Soc. 
Conf. Computer Vision and Pattern Recognition, page 210217, 2001. 1 

[5] M. Cho, J. Lee, and K. M. Lee. Reweighted random walks for graph matching. In Proceedings of the 11th Eu- 
ropean conference on Computer vision: Part V, ECCV'IO, pages 492-505, Berlin, Heidelberg, 2010. Springer- 
Verlag. 1, 2, 6, 7, 8 

[6] T. Cour, R Srinivasan, and J. Shi. Balanced graph matching. In NIPS'06, pages 313-320, 2006. 1, 2, 6 

[7] C. Davis and W. M. Kahan. Some new bounds on perturbation of subspaces. Bull. Amer Math. Soc, 75(4):863- 
868, 1969. 13 

[8] A. Egozi, Y. Keller, and H. Guterman. A probabilistic approach to spectral graph matching. IEEE Transactions 
on Pattern Analysis and Machine Intelligence, 99(PrePrints), 2012. 1 

[9] D. Emms, R. C. Wilson, and E. R. Hancock. Graph matching using the interference of continuous-time quantum 
walks. Pattern Recogn., 42(5):985-1002, May 2009. 1, 2 



[10] M. Eshera and K. Fu. A graph distance measure for image analysis. IEEE Trans. Syst. Man Cybern., page 
398408, 1984. 2 

[11] S. Gold and A. Rangarajan. A graduated assignment algorithm for graph matching. IEEE Trans. Patt. Anal. 
Mach. IntelL, 18, 1996. 2, 6 

[12] M. Gori, M. Maggini, and L. Sarti. Exact and approximate graph matching using random walks. IEEE Trans. 
Pattern Anal. Mach. IntelL, 27(7): 1 100-1 111, July 2005. 1, 2, 5, 6 

[13] D. K. Hammond, P. Vandergheynst, and R. Gribonval. Wavelets on graphs via spectral graph theory. Applied 
and Computational Harmonic Analysis, 30(2): 129-150, Mar 2011. 3 

[14] S. Jouili and S. Tabbone. Graph matching based on node signatures. In Proceedings of the 7th IAPR-TC-15 
International Workshop on Graph-Based Representations in Pattern Recognition, GbRPR '09, pages 154-163, 
Berlin, Heidelberg, 2009. Springer- Verlag. 2, 5, 6 

[15] H. Kuhn. The hungarian method for the assignment problem. Naval Research Logistic Quarterly, page 8397, 
1955. 5 

[16] M. Leordeanu and M. Hebert. A spectral technique for correspondence problems using pairwise constraints. 
In Proceedings of the Tenth IEEE International Conference on Computer Vision - Volume 2, ICCV '05, pages 
1482-1489, Washington, DC, USA, 2005. IEEE Computer Society. 2, 6 

[17] M. Leordeanu, M. Hebert, and R. Sukthankar. An integer projected fixed point method for graph matching and 
map inference. In Proceedings Neural Information Processing Systems. Springer, December 2009. 2, 6 

[18] B. Luo and E. R. Hancock. Structural graph matching using the em algorithm and singular value decomposition. 
IEEE Trans. Pattern Anal. Mach. IntelL, 23(10): 1120-1 136, Oct. 2001. 1, 2 

[19] M. J. Lyons, J. Budynek, and S. Akamatsu. Automatic classification of single facial images. IEEE Trans. Pattern 
AnaL Mach. IntelL, 21(12):1357-1362, Dec. 1999. 1 

[20] D. Maio and D. Maltoni. A structural approach to fingerprint classification. In Proceedings of the International 
Conference on Pattern Recognition (ICPR '96) Volume Ill-Volume 7276 - Volume 7276, ICPR '96, pages 578-, 
Washington, DC, USA, 1996. IEEE Computer Society. 1 

[21] H. Qiu and E. R. Hancock. Graph matching and clustering using spectral partitions. Pattern Recogn., 39(1):22- 
34, Jan. 2006. 2 

[22] A. Robles-Kelly and E. R. Hancock. String edit distance, random walks and graph matching. In Joint lAPR 

International Workshop on Structural, Syntactic, and Statistical PR, pages 104-112, 2002. 2 

[23] A. Robles-Kelly and E. R. Hancock. Graph edit distance from spectral seriation. IEEE Trans. Pattern AnaL 
Mach. IntelL, 27(3):365-378, Mar. 2005. 2 

[24] C. Schellewald and C. Schnorr. Probabilistic subgraph matching based on convex relaxation. In Proceedings of 
the 5th international conference on Energy Minimization Methods in Computer Vision and Pattern Recognition, 
EMMCVPR'05, pages 171-186, Berlin, Heidelberg, 2005. Springer- Verlag. 2 

[25] A. Shokoufandeh and S. Dickinson. Applications of bipartite matching to problems in object recognition. Pro- 
ceedings, ICCV Workshop on Graph Algorithms and Computer Vision, September 1999. 2, 5 

[26] A. Shokoufandeh and S. J. Dickinson. A unified framework for indexing and matching hierarchical shape struc- 
tures. In Proceedings of the 4th International Workshop on Visual Form, lWVF-4, pages 67-84, London, UK, 
UK, 2001. Springer- Verlag. 2, 3 

[27] J. Sun, M. Ovsjanikov, and L. Guibas. A concise and provably informative multi-scale signature based on heat 
diffusion. In Eurographics Symposium on Geometry Processing (SGP), 2009. 1, 3 

[28] M. Taj, E. Maggio, and A. Cavallaro. Multi-feature graph-based object tracking. In Proceedings of the 1st 
international evaluation conference on Classification of events, activities and relationships, CLEAR' 06, pages 
190-199, Berlin, Heidelberg, 2007. Springer- Verlag. 1 

[29] A. Torsello and E. R. Hancock. Learning structural variations in shock trees. In Proceedings of the Joint lAPR 
International Workshop on Structural, Syntactic, and Statistical Pattern Recognition, pages 113-122, London, 
UK, UK, 2002. Springer- Verlag. 1 

10 



[30] S. Umeyama. An eigendecomposition approach to weighted graph matching problems. IEEE Trans. Pattern 
Anal. Mack IntelL, 10(5):695-703, 1988. 1, 2 

[31] J. van den Heuvel and S. Pejic. Using laplacian eigenvalues and eigenvectors in the analysis of frequency 
assignment problems. Annals of Operations Research, 107(l-4):349-368, 2001. 11 

[32] B. J. van Wyk and M. A. van Wyk. A pocs-based graph matching algorithm. IEEE Trans. Pattern Anal. Mach. 
IntelL, 26(11):1526-1530, Nov. 2004. 2, 6 

[33] M. A. van Wyk, T. S. Durrani, and B. J. van Wyk. A rkhs interpolator-based graph matching algorithm. IEEE 
Trans. Pattern Anal. Mach. IntelL, 24(7):988-995, July 2002. 1 

[34] R. C. Wilson and P. Zhu. A study of graph spectra for comparing graphs and trees. Pattern Recogn., 41(9):2833- 
2841, Sept. 2008. 1 

[35] P. C. Wong, H. Foote, G. Chin, P. Mackey, and K. Perrine. Graph signatures for visual analytics. IEEE transac- 
tions on visualization and computer graphics, 12(6): 1399^13, 2006. 2, 6 

[36] M. Zaslavskiy, F. Bach, and J. -P. Vert. A path following algorithm for the graph matching problem. IEEE 

Transactions on Pattern Analysis and Machine Intelligence, 31(12):2227-2242, 2009. 2 

[37] R. Zass and A. Shashua. Probabilistic graph and hypergraph matching. CVPR, 2008. 6 

[38] G. Zhao, B. Luo, J. Tang, and J. Ma. Using eigen-decomposition method for weighted graph matching. In Pro- 
ceedings of the intelligent computing 3rd international conference on Advanced intelligent computing theories 
and applications, ICIC'07, pages 1283-1294, Berlin, Heidelberg, 2007. Springer- Verlag. 2 

A Proof of Theorem 1 (LFS Continuity) 

Theorem (LFS Continuity). Let C be the laplacian of a graph, and C be a perturbed laplacian. Let the size of the 
graph be n, and Xi < ■ ■ ■ < Xk denote the k distinct eigenvalues of C. Let Si{t) and s^(i) denote the LFS's of node i. 
IfXj+i - Xj > S,yj, \\C - C'Wf < (■ < 6, and h{-; •) e C^(M^), then ifk = n, we have 

where Co{S, t) is a constant independent of e, and if k < n we have 

\sm - s'M < at) (^^ - 1) + C2{t)e, 

where Ci (t), C2 (t) are constants depends only on t. 

Proof of the first part of the theorem for non-repeated eigenvalues (fc = ri) requires the following theorems. 

Theorem 3. [ ] (Sensitivity of Generalized Eigenvalues and Eigenvectors) Let Kq, Mq be symmetric positive 
definite with distinct generalized eigenvalues Aoi and generalized eigenvectors xoi, Le. i^o^Oi — Aoi-MoXoi, Vi, and 
Xfj^MoXoj = 5f, Vi, j, where Sj is the Kronecker delta. If K = Kq + AK and M = Mq + A Af are symmetric and 
positive definite, then the generalized eigenvalues Xi and eigenvectors x,j of K:x.i — XiMXi satisfy 

1. X, = Ao, + ^J,{AK - Ao,AAf)xo,. 

2. x, = xoj(1 - 2X(JiAMxoi) + E^=i — — x^^-^A^- xoj. 

Theorem 4. [3 1 ] (Maximum Eigenvalue) Let Xmax be the largest eigenvalue of a connected weighted graph G. Let 
w{u, v) be the weight for edge {u, v), and d{v) — X^u^^d w{u, v) the sum of incident weights. Then 



Xmax < max <^ d{v) H 

vev{G) [ d{v) 



Now let's prove the first part of the Theorem. 
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Proof. LFS Continuity For Non-Repeated Eigenvalues Laplacian eigenproblem is a special case of the generalized 

eigenproblem in Theorem 3. If we let M^ = I, AM = 0, and Ai^ = AC = C - C . Let (Aj, 0,), (A-, </)■) be the 
eigenpairs of C and C . By Theorem 3 we have. 



A' - A,, 






E 



A£(j), 






Note AC is also a graph laplacian, and from Theorem 4, we have 

d{v) 



Xmaxi^L) < max <{ d{v) 



Then, we have 



< max {d{u) + d{v)} < \AL\f < e 

{u,v)GE 



|A: - A,| = |0^A/:0,| < X^aA^C) < e 

N i,T, 



Oi\\2 = 



< 



IE' 



A/:'^z 

min(Ai — Xj) 



Pil|2 



iE< 



'il|2 



< 



For /i(t; •) G C^(]R^), ft. has a Taylor expansion at A^ 

hit; A^) = /i(t; AO + ft,(t)(A^ - A,) + OlAA^). 
Plug (1) into Si{t) and ignoring higher order terms, we have 



WM~^^{t)\ = 



Y^h{t;Xk)Mi)^ ~Y.^{t;>^kWs? 



< 



Y, hit; Afe) {M^r " M^') E '^'^ W(^^ ^ ^"^-^UO 



maxfc(ft,(t, Afe))2|V| ,, , ,,,,^, ^ ,^ , 

< ^ , ^^ ' ' e + max(fefc(t))|F|£^Co((5,^)e 

fe 



The proof to the second part of the theorem requires the following prerequisite. 



(1) 



n 



Lemma 1 (Subspace Invariance of LFS). Let Cbe a graph laplacian, {Xi} are the eigenvalues of C with multiplicity 
{ki }. Suppose 4>ii,- ■ ■ , (t>iki, ond 0^j^ , • • • , (jj'^f, . are two sets of orthogonal eigenvectors corresponding to Xi, si (t) and 
s[{t) are the LFS for node I computed using the two sets of eigenvectors respectively. Then, we have 

si{t)^s'i{t),\fl 

Proof. For the case of eigenvalues with multiplicity, LFS of node I could be written as 
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WLOG, we only need to consider a specific \j with multiplicity kj, and all we need to show is that VZ, the following 
is true 



J2Ml)'^J2^'s^(l)' 



(2) 



1=1 



4=1 



Note if we let $ = [^^i, • • • , 4ijkj], and $' 



f'jir 



'jk, 



, (2) is equivalent to 



diag($$^) =diag($'$'^) 

As $ and $' span the same subspace, $' can be expressed as a rotation (possibly reflection) and the permutation of $, 
i.e. 3R E O'^ and P E S'' a permutation matrix, such that $' = ^RP. Hence, we have 



$'$'T = ^RPp'^R^q>'^ = $$T 



n 



Theorem 5. [I ] (Perturbation on Eigenvalues) Let C be the laplacian of a graph with n nodes, C = C + A£ is a 
perturbed graph laplacian. If Xi < ■ ■ ■ < Xn ond X[ < ■ ■ ■ < X'^^ are the eigenvalues to C and CJ respectively. Then 
Vi, we have 

|A,-A:|<||A£||2 

Tlieorem 6. [ ] (Davis, Kalian sin theta theorem) LetA,A + UE M"""" be symmetric, E^.FqE M"""", E^.F^e 
]fjnx(n-fc) ^/^g^g ^ ^ {Eq\Ex\, F = [Fo\Fi] E M"^" are orthogonal basis q/M". Let 



A 

A + H 



^1 1 X)^^ 



Let 



dpiEo,Fo) = \\EqEq - FqFq II2 
IfX{Ao) C [a, b], X{Bi) C (-00, a-d)U{b + S, 00). Then 

d,{Eo,Fo)<^\\H\\2. 

Finally, we will have our continuity theorem on graph HKS. 

Proof. LPS Continuity For Repeated Eigenvalues 

Let di be the multiplicity of A^ for Vi, and {(pn, ■ ■ ■ , (l)idi}i be the set of eigenvectors. Let A'j^ < • • • < AJj be the 
eigenvalues of L' and {(p'^} be the corresponding eigenvectors. Clearly, we could rewrite {A^} as {X'n, ■ ■ ■ , AJ^ }i, 
and {v^} as {(p'^, • • • , 4>'id}i f°i" mathematical convenience. Then the difference on two LFS can be written as 



\sm-m\^ 



^^(Mt;A,)(/.,fe(^f-Mt;A;,)</'' 



3 k 



< 



E 



^(Mi;A,)0,/c«'-Mi;A;-fe)0',fe« 
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Substituting (1) and ignoring higher order terms, we will have 



<Ht;Xj) 



J2(.^M^)'-<l^',k{^' 



h,it)\\AChJ2v' 



2 

jki' 



The last term used the fact in Theorem 5. If we let <l>j = [(pji, • • • , <pjdj] and $' = W jii ' ' ' i 4*' jd]^ ^i^d diag(-)i be 
the i''' element of the diagonal of a matrix, we could rewrite 



J2{4M^f-4>',,{if 



diag($j$J)^-diag($;$'J)^ 
diag($,cl>7-$;.$'J)^ 



< 



3 J 3 3 



max x^ ( $,$7 - $'.*&' 



Ibll2=|y||2= 

<v^l|A/:||2 

d — e 



Jjy 



for t < 5, where e^ is the standard vector. Note that || • II2 < || • ||f> and summing over all j, we then have 

\sm-s'At)\<Y.(h{t;\,)-^J]+Y,h,{t)e 



at) 



S- 



l]+C2{t)e 



n 



B Proof of Theorem 2 (Noise Tolerant Approximation) 

Theorem (Noise Tolerant Approximation). Let d'^{i,j, a, b) and d'^{i,j, a, b) be the pairwise heat kernel distance 
and pairwise adjacency distance for graph G and G'. Let Xi and X'^ be the set of eigenvalues ofG and G' . 



lim ^lMM ^ ,A 
t->-o t 



d^{i,j,a,b), 



and 



J2J2('^fizJ,a,b)f < ^exp(-2a0 +^exp(-2iA'J 

Proof Let JCt — exp(— 1£) be the matrix form of heat kernel, first order Taylor expansion will give 

JCt ^ I - tc + 0{f) 

Therefore, for JCt and K,[, we have 

]Ct~JC[ = -t{C-C') + 0{t^) 

= -t{D - D') + t{A - A') + 0{t^). 
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which is equivalent to 



On the other hand, 



hm ^d 2,j,a,6) 

t->-o t 



5]^(df(^,J,a,fe))2<||/C,-/C;|b 

<ll/Ct||F + i|/c;i|p 

fc k 

= Y^ exp(-2tAfe) + Y^ exp(-2iAfe) 



D 
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